3.11.5.9, not 3.11.5.8

3.11.5.9, not 3.11.5.8

Yeah, it was hanging there for a long time, just updated it to 3.11.5.9

@Bouncheck , WDYT should we remove these tests ? IMHO this has to be tested on the driver side and other part of this issue is handled in AlterTabletIT.java

I see no reason to remove BaseScyllaIntegrationTest.
If all invariants in Driver3WorkerCQLIT were broken then we should have a clear understanding why it's ok now.
For broken invariants there is no need to fix their tests. I'd mark them as ignored, document and remove later on.
If there are new important invariants then those should have new tests.

Of course if something is guaranteed not to change on the driver side then it does not necessarily need a test in cdc library.

I see no reason to remove BaseScyllaIntegrationTest. If all invariants in Driver3WorkerCQLIT were broken then we should have a clear understanding why it's ok now. For broken invariants there is no need to fix their tests. I'd mark them as ignored, document and remove later on. If there are new important invariants then those should have new tests.

It was broken because metadata was not changing because of the following logic:

                      ColumnDefinitions newMetadata = null;
                      if (rows.metadata.metadataId != null) {
                        newMetadata = rows.metadata.columns;
                        assert statement instanceof BoundStatement;
                        BoundStatement bs = (BoundStatement) statement;
                        bs.preparedStatement().getPreparedId().resultSetMetadata =
                            new PreparedId.PreparedMetadata(
                                rows.metadata.metadataId, rows.metadata.columns);
                      }
                      MultiPage.this.nextPages.offer(new NextPage(newMetadata, rows.data));

It was broken the whole time.

I wonder if with the driver change this part below should be changed somehow or not.

I wonder if with the driver change this part below should be changed somehow or not.

scylla-cdc-java/scylla-cdc-driver3/src/main/java/com/scylladb/cdc/cql/driver3/Driver3WorkerCQL.java

Lines 153 to 162 in 02068f7

	Row row = rs.one();
	if (schema == null) {
	try {
	schema = Driver3SchemaFactory.getChangeSchema(row, session.getCluster().getMetadata());
	} catch (Driver3SchemaFactory.UnresolvableSchemaInconsistencyException ex) {
	fut.completeExceptionally(ex);
	return;
	}
	}
	Driver3RawChange newChange = new Driver3RawChange(row, schema);

It seems like we may not be fully leveraging the "skipMetadata" improvements if we do not change it.
In this snippet the schema field is set only once per PreparedStatement. IIUC it will not be in sync with contents of the ResultSet if an ALTER operation is performed during reading

Yes exactly, that change was missing in the PR, for it is already too late, issue is fixed and metadata updates are working.

The tests for old invariants should be removed, not adjusted to pass: This invariant will not be true anymore after driver update:

   public void testPreparedStatementSameSchemaBetweenPages() throws ExecutionException, InterruptedException, TimeoutException {
        // Tests an assumption used in the implementation
        // that given a PreparedStatement, it will always
        // return the rows with the same schema - in this
        // case: when a schema change happens between
        // pages of a query.
        // Note that Driver3Session deliberately sets
        // a protocol version to V4, while these
        // tests do not. This is on purpose:
        // if Scylla in the future implements a
        // support for new protocol version, this
        // test might fail and you (are you investing
        // this failure now?) should decide how to proceed.
        // See Driver3WorkerCQL for more context.

and this too:

    public void testPreparedStatementOldSchemaAfterAlter() throws ExecutionException, InterruptedException, TimeoutException {
        // Tests an assumption used in the implementation
        // that given a PreparedStatement, it will always
        // return the rows with the same schema - in this
        // case: when a query is prepared, then a schema
        // change happens and afterwards the query is started
        // it will still return the results with the
        // schema as of the time of preparing the query.

Alternatively we could have a test that the schema is fresh between pages. Maybe one day we'll change to full protocol v5 without patches and this would guard against a mistake during transition.

I have dropped the whole suite, please take a look.